Replace Python tests with GO test #45503

khushijain21 · 2025-07-22T14:15:05Z

Proposed commit message

This PR moves Python tests to GO

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

github-actions · 2025-07-22T14:15:16Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

mergify · 2025-07-22T14:15:46Z

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @khushijain21? 🙏.
For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
backport-active-all is the label that automatically backports to all active branches.
backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

libbeat/testing/integration/output_watcher.go

rdner · 2025-07-24T12:10:41Z

We should rename and split filebeat/testing/integration/sample_test.go into meaningful test files. e.g. cmd_test.go, cmd_modules_test.go, etc. It should not all be in one giant file.

libbeat/testing/integration/output_watcher.go

libbeat/testing/integration/integration.go

elasticmachine · 2025-07-25T08:03:08Z

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

khushijain21 · 2025-07-25T12:29:59Z

filebeat/tests/system/test_publisher.py

-
-    def test_registrar_file_content(self):
-        """
-        Check if registrar file is created correctly and content is as expected


The existence of registry and its contents is already tested by
https://github.com/khushijain21/beats/blob/onweek1/filebeat/tests/integration/filestream_truncation_test.go#L70

filebeat/testing/integration/cmd_module_test.go

filebeat/testing/integration/container_test.go

mauri870

I left a few minor comments. Thanks for your work on this and for cleaning up the tests!

filebeat/testing/integration/container_test.go

rdner

Great to see the framework evolving as we see more test-cases.

There are a few minor comments. I would approve after the got addressed.

rdner · 2025-07-31T12:18:24Z

filebeat/testing/integration/cmd_test.go

+
+	config := `
+filebeat.inputs:
+  - type: log


I'd encourage to test the filestream input only because log is deprecated.

Also, there is known issue with --once #33718

rdner · 2025-07-31T12:19:57Z

filebeat/testing/integration/container_test.go

+
+	config := `
+filebeat.inputs:
+- type: container


Let's test filestream with the container parser instead as described here https://www.elastic.co/docs/reference/beats/filebeat/filebeat-input-container.

The container input is now deprecated.

rdner · 2025-07-31T12:21:29Z

filebeat/testing/integration/container_test.go

+	}
+}
+
+type registryEntry struct {


Everything registry should be in a separate file and be available to all the tests as a set of helper functions.

rdner · 2025-07-31T12:22:30Z

filebeat/testing/integration/deprecated_test.go

+
+		config := `
+filebeat.inputs:
+  - input_type: log


Should be another test for the container input as well. Expect the same behavior.

rdner · 2025-07-31T12:26:08Z

filebeat/tests/integration/crawler_test.go

+}
+
+// counts number of lines in the given file and  asserts if it matches expected count
+func CountLinesInFile(t *testing.T, path string, count int) {


This should be near the integration.GenerateLogFile function as a part of the framework.

rdner · 2025-07-31T12:30:07Z

libbeat/testing/integration/output_watcher_test.go

+		InputDoc := mapstr.M{
+			"message": "sample dummy message",
+			"fields": mapstr.M{
+				"number": 20,
+				"slice":  []string{"20", "30", "40"},
+			},
+			"isthisboolean": false,
+			"nilValue":      nil,
+		}


I think it's better to have raw JSON here, then parse it to mapstr.M and test the whole cycle. So, we're as close to the real use-case as possible.

rdner · 2025-07-31T12:31:24Z

libbeat/testing/integration/integration.go

+}
+
+// GetTempDir implements the BeatTest interface.
+func (b *beatTest) GetTempDir() string {


This is great!

Remove TestCmd Python Test

c64e103

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 22, 2025

mergify bot assigned khushijain21 Jul 22, 2025

Merge branch 'main' into onweek1

746ac82

khushijain21 changed the title ~~Remove TestCmd Python Test~~ Replace Python tests with GO test Jul 22, 2025

add 2 more tests

1c54aeb

khushijain21 marked this pull request as ready for review July 23, 2025 15:48

Merge branch 'main' into onweek1

43da8a9

khushijain21 requested a review from a team as a code owner July 23, 2025 15:48

khushijain21 requested review from mauri870, faec and rdner July 23, 2025 15:48

khushijain21 added 2 commits July 23, 2025 21:19

add tag

694bd73

add correct tag

e2aa55b

khushijain21 changed the title ~~Replace Python tests with GO test~~ [WIP] Replace Python tests with GO test Jul 23, 2025

rdner reviewed Jul 24, 2025

View reviewed changes

libbeat/testing/integration/output_watcher.go Outdated Show resolved Hide resolved

khushijain21 added 4 commits July 24, 2025 18:42

add test_crawler

d74aa47

Address comments

5d41437

add more test_crawler

064b0ca

fix cli

76ea1ce

rdner reviewed Jul 24, 2025

View reviewed changes

libbeat/testing/integration/output_watcher.go Outdated Show resolved Hide resolved

libbeat/testing/integration/output_watcher.go Show resolved Hide resolved

libbeat/testing/integration/integration.go Show resolved Hide resolved

pierrehilbert added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Jul 25, 2025

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jul 25, 2025

khushijain21 added 3 commits July 25, 2025 15:06

add reload tests

b9924fe

add publish

279117c

add test case

0abeaca

khushijain21 commented Jul 25, 2025

View reviewed changes

khushijain21 added 2 commits July 25, 2025 18:11

fix ci

7002433

add container tests

b6da401

khushijain21 changed the title ~~[WIP] Replace Python tests with GO test~~ Replace Python tests with GO test Jul 25, 2025

khushijain21 requested a review from rdner July 25, 2025 15:56

khushijain21 added 14 commits July 25, 2025 21:26

Merge branch 'main' into onweek1

ff9bc7d

fix make chek

7570362

add docker.log

af9ac12

add filebeat once test

e702979

reset reload input

3d85af2

remove pytest

54b4ead

make check

879642e

add cmd module file

c5d485b

fix cmd_module_test.go

77f3cc2

fix integ test

8d20d06

Merge branch 'main' into onweek1

41077f3

remove reload tests

59371d7

fix ci

89e5ab2

no new variables

89ded19

mauri870 reviewed Jul 29, 2025

View reviewed changes

filebeat/testing/integration/cmd_module_test.go Outdated Show resolved Hide resolved

mauri870 reviewed Jul 29, 2025

View reviewed changes

filebeat/testing/integration/container_test.go Outdated Show resolved Hide resolved

mauri870 approved these changes Jul 29, 2025

View reviewed changes

add t.Context()

e3dd6df

rdner reviewed Jul 30, 2025

View reviewed changes

filebeat/testing/integration/container_test.go Show resolved Hide resolved

address review comments

330bfec

khushijain21 requested a review from rdner July 31, 2025 06:00

Merge branch 'main' into onweek1

0b57270

rdner reviewed Jul 31, 2025

View reviewed changes

Replace Python tests with GO test #45503

Are you sure you want to change the base?

Replace Python tests with GO test #45503

Uh oh!

Conversation

khushijain21 commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Uh oh!

github-actions bot commented Jul 22, 2025

🤖 GitHub comments

Uh oh!

mergify bot commented Jul 22, 2025

Uh oh!

Uh oh!

rdner commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Jul 25, 2025

Uh oh!

khushijain21 Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mauri870 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rdner left a comment

Choose a reason for hiding this comment

Uh oh!

rdner Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

rdner Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

rdner Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

rdner Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

rdner Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

rdner Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

rdner Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

khushijain21 commented Jul 22, 2025 •

edited

Loading

rdner commented Jul 24, 2025 •

edited

Loading

khushijain21 Jul 25, 2025 •

edited

Loading